Create LLM Resources

LLM resources provide a centralized way to manage your Artificial Intelligence (AI) model connections within Druid. By creating LLM resources in the Druid Portal, you define the necessary credentials and endpoints once, allowing you to reuse them consistently across multiple features, including:

  • LLM NLU (Natural Language Understanding)
  • LLM Machine Translation
  • LLM Response Streaming

This unified approach simplifies configuration and ensures consistent access to your language models throughout your bot implementation.

NOTE: This feature is available in Druid 9.11 and higher.

Different providers support different configuration options. For example, Azure OpenAI supports two authentication methods:

  • API key
  • OAuth (Microsoft Entra ID)

Other providers support only API keys.

NOTE: OAuth for Azure OpenAI resources is available starting with Druid version 9.18.

Prerequisites for using OAuth for Azure OpenAI resources

  1. Register an application in Microsoft Entra ID following the Microsoft instructions.
  2. After you register the application, note the following values:
    • Application (client) ID
    • Directory (tenant) ID

    You can find these values in the application Overview page.

  3. Create a client secret for your app (Manage > Certificates & Secrets). When a new client secret is generated, copy the value as you'll need it in Druid.

  4. Grant the application access to Azure OpenAI:
    1. Open your Azure OpenAI resource in the Azure portal.
    2. Select Access control (IAM).
    3. Select Add role assignment.
    4. Assign the role Cognitive Services OpenAI Contributor.
    5. Select Members.
    6. Choose Service principal.
    7. Select the application you registered.
    8. Select Review + assign.

Add an LLM resource

To add and configure a new LLM connection:

  1. Go to Administration > LLM Resources.
  2. Click the Create LLM Resource button. The LLM Resource Details modal appears.
  3. Complete the LLM resource details:
    1. Client Type: Select the type of LLM client you are using.
    2. Model: Select or enter the specific model name you intend to use.
    3. API Url: Enter the LLM service endpoint URL.
    4. API key: Enter the secret API key provided by your LLM service provider.

      5. If you selected Azure OpenAI as Client Type, two authentication options are available:

      • API Key
      • OAuth (AAD). Select OAuth (AAD) for enhanced security. Unlike a static API key, which is valid until rotated, OAuth uses short-lived tokens. When selected, you must provide the following values you copied from Azure:

        • Azure TenantId: The unique identifier of your Azure instance (the Directory (tenant) ID).

        • Azure App ClientId: The Application ID of your registered app (the Application (client) ID).

        • Azure App Secret: The client secret value generated for your app.

    HINT: If you are using a Druid LLM resource (a model provided by Druid), you can find the necessary API Url and API key in the Druid Portal under Administration > Druid Services.

  4. If you selected Google as Client Type, you also need to enter the location of the location for a Vertex AI and your Google project id.
  5. Click Save & Close.